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A system for providing an auditory display of two- 
dimensional patterns as an aid to the blind includes a 
scanning device, such as a slow scan television cam- 
era, for producing first and second voltages respec- 
tively indicative of the vertical and horizontal posi- 
tions of the scan and a further voltage indicative of the 
intensity at each point of the scan and hence of the 
presence or absence of the pattern at that point. The 
vertical position of the scan is represented aurally by 
the frequency of the tone heard, the first voltage being 
used to control the frequency output of a VCO. The 
horizontal position of the scan is represented aurally 
by the interaural difference between two tones pres- 
ented so that a point on the left side of a pattern is in- 
dicated by a relatively loud sound in the left ear. The 
voltage related to scan intensity controls transmission 
of the sounds to the subject so that the subject knows 
that a portion of the pattern is being encountered by 
the scan when a tone is heard, the subject determining 
the position of this portion of the pattern in space by 
the frequency and interaural difference information 
contained in the tone. 

15 Claims, 5 Drawing Figures 
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1. * ' 2 . 

AUDITORY DISPLAY FOR THE BLIND This coding system has proved, to be very natural and 

rnwcDNfu/fCM . easv to learn ' a number of subjects having mastered it 

GOVERNMENT RIGHTS by listening to tape recordings lasting only one half 

hour. 

The invention described herein was made under a 5 According to presently . preferred embodiments 

grant of the United States Government and may be thereof, the auditory display system of the invention, 

manufactured and used by or for the Government for thc pattern ^ scanne d using an oscilloscope, television 

governmental purposes without the payment of any camera or the like and voltage signals corresponding to 

royalties thereon or therefor - a „ — ^- the horizontal and vertical position of the scan are de- 

nAri^rDhiiMnrtPTun iw\/cm™™ 10 rived ' ^ Uvertical " voltage signal, preferably after. 

BACKGROUND OF THE INVENTION shaping using a function generator, is used to drive a 

There are about one-half million blind people in the voltage controlled oscillator which thus produces a fre- 

United States at the present time and a great time of re- quency output in accordance with the vertical position 

search has been focussed on finding ways to aid the of the scan. The function generator is used to cause the 

blind in "reading" and "seeing.** In addition to more 15 frequency to vary in an exponential manner with height 

familiar techniques such as Braille, substantial work since, to a first approximation, the ear responds to the 

has been done in presenting printed materials to the logarithm of frequency. The "horizontal" voltage is 

blind in the form of coded sounds. Generally speaking, used to control the amplitude of the sound heard by the 

devices of this type, often called optophones, convert subject in accordance with the horizontal position of 

a light image of the material to be "read" into a corre- 20 the scan and, in accordance with a preferred embodi- 

sponding electrical signal using optical-electronic ment, is applied to first and second function generators 

transducers such as photo-cells. This electrical signal is connected respectively to the left and right earphones 

used to control the output of a variable frequency oscil- of the headset referred to above, one of the inputs 

lator or oscillators so that a distinctive tone is gene r- being inverted beforehand. The function generators 

ated. There are numerous examples of such devices in 23 preferably comprise variable diode function generators 

the patented art and reference is made to U.S. Pat. Nos. each of which generates a roughly exponential function 

2,412,467 ( Morton); 2,487,51 1 (Bedford); 2,582,728 of m e horizontal position of the scan. A pair of analog 

(Walker); 2,615,992 (Flory et al.); 3,007,259 (AJ>ma multipliers are used to make the audio outputs of this 

et al.); 3,229,075 (Palti); 3.359,425 (Smith); and ■ variable frequency oscillators follow the exponential 

3,676,938 (Trehub) for examples of aural "reading" JV waveforms generated by the function generators and 

systems. Although it would not be worthwhile to dis- the resu i tant outputs are applied to the left and d ht 

cuss the disadvantages of individual systems in any de- earphones 

tail, in general, the systems range from those which suf- A signal porporliona i t0 intensity of the scan 

fer from an inability to represent any patterns other is used to control the input to the headphones so that 

than the very simplest, on the one hand, to those which nQ sound is heard when the gcan h out of ister wUh 

the coding used in identifying etters or shapes is so the m This u ferabI accornplished by elec . 

complex that learning is d.ff.cult, on the other Other tfonic swHch connectcd m series with the voh ' e con . 

disadvantages include complexity, high cost and main- tro , |ed Md contro „ ed by an intensity signal 

tenance difficulties. 4Q derived from the television camera or a control signal 

SUMMARY OF THE iNVENTlON derived from a photomultiplier tube positioned to re- 

. , . , . . . .. . ceive light from the scanning dot of the oscilloscope. 

In accordance with the invention, an auditory display c ■ u *v . ' j j * • -.. 

\ c r 7 Further features and advantages of the invention will 

system is provided which eliminates the need for the , . f . • c A . . . , . 

complex coding employed in many prior.art systems ^ «t fo^ ,n or apparent from the deU.led descr.p- 

and which eliminates, or reduces the effect of, other 45 "on of the preferred embedments found here.nbelow. 
problems associated with prior art systems. 

In general, the system of the invention provides a BRIEF DESCRIPTION OF THE DRAWINGS 

two-dimensional spatial representation of the image " _ — . 

pattern to be "viewed." The pattern is scanned using 3Q FIG. 1 * s a schematic circuit diagram, in block form, 

raster or line scanning and the X-Y position of the scan of an auditory display system in accordance with a first 

is indicated aurally. The subject wears binaural head- embodiment of the invention; 

phones and the horizontal position of the scan is FIGS. 2 and 3A and 3B are waveforms associated 

"coded" using differences in the interaural intensity of w *Jh the system of FIG. 1 used in explaining the opera- 
the sounds heard by the subject, that is, by varying the . 55 tion of that system; and 

amplitude of the sounds heard by the subject in his left FIG- 4 is a block-form schematic diagram of a second 

and right ears in accordance with the horizontal posi- embodiment of the invention, 

tion in the scan. The vertical position of the scan is DESCRIPTION OF THE PRFFFRRFD 

coded by frequency, higher positions of the scan pro : EMBODIMEN-re 

ducing higher audio frequencies and lower positions 60 cMlJODIMbrN 

producing lower frequencies. Thus, with the scan lo- Referring to FIG. 1, a first embodiment of the audi- 

cated in the upper left hand corner of the field to be tory display system of the invention is shown. The sys- 

scanned, a high pitched sound which would be loudest tern uses "flying spot" scanning techniques to scan the 

in the left ear would be heard, which sound, because of pattern to be displayed and to this end an oscilloscope 

the coding, wbyld appear to be coming a corresponding $ 5 10 is provided which scans an image pattern indicated 

position in space. To present a predetermined pattern . at 12. In experimental use of the invention the pattern 

to a subject the sound is transmitted only when the scan used has been formed by pieces of cardboard which are 

crosses the pattern so that the subject can construct the pasted or otherwise affixed to the screen of the oscillo- 

pattern spatially in his mind in accordance with the scope. The vertical and horizontal position of the oscil- 

seeming locations of the sounds, and their durations. ' loscope scan, i.e., the position of the dot of light on the 
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oscilloscope screen, are respectively determined by the" to obtain the waveforms which are actually heard by 
output voltages from a vertical sawtooth generator 14 the subject, the outputs of multipliers 38 and 40 being 
and a horizontal sawtooth generator 16. For a scan of connected through a dual channel audio amplifier 42 
twenty horizontal lines, the frequency of horizontal to the left and right earphones of a headphone set 44. 
sawtooth generator 16 is made to be twenty times that 5 The audio waveforms, as a function of horizontal posi- 
of the vertical sawtooth generator 14. tion, for the signals transmitted to the left and right ears 

It will be appreciated that light from the moving dot are shown in FIGS. 3A and 3B. As illustrated, the 
on the screen of oscilloscope 10 will be blocked by sound volumes are equal at the center of the scan. In 
image pattern 12 which behind this pattern and will be an exemplary embodiment, the sound volumes change 
transmitted for areas outside of pattern 12. A photo- 10 40db from one side to the other and the frequency var- 
multiplier tube 18 is used to detect the transmitted ; ies from 5,000 Hz at the top of the scan to 200 Hz at 
light, which is focussed thereon by a suitable lens sys- the bottom. Sine, triangular and square audio wave- 
tern indicated by lens 20, and to produce an output forms have been used and found to be satisfactory, 
voltage in accordance therewith. The output of photo- The output of level sensor 24 can also be used to siow 
multiplier tube 18, after amplification by amplifier 22, 15 the scan rate when the oscilloscope beam, i.e., scanning 
is applied in the input of a level sensor 24. Level sensor dot, is behind the pattern and, to this end, this output 
.24 controls switching of an electronic switch 26 such is connected to vertical and horizontal sawtooth gener- 
that when there is no light input to photomultiplier 18, ators 14 and 16. This arrangement permits rapid scan- 
corresponding to the dot of light from the oscilloscope ning when no sound is transmitted to the subject and 
10 being blocked by pattern 12, switch 26 is turned on 20 slower scanning when a portion of the pattern 12 is en- 
and, as explained hereinbelow, sound is permitted to countered. The changing of sweep rates allows patterns 
reach the subject. When light is detected, switch 26 is to be scanned more quickly and has not been found to 
turned off, resulting in silence. be confusing to subjects. It will be appreciated the out- 

In addition to controlling the horizontal motion of lines of the pattern shapes can be presented much more 
the dot of light on the screen of oscilloscope 10, hori- 25 rapidly with this approach than where the scanning dot 
zontal sawtooth generator 16 is used as an input to first goes slowly from one side of the figure to the other. In- 
and second function generators 28 and 30, Function stead, the dot seems to jump from one side to the other, 
generators 28 and 30 preferably comprise variable waiting a few tens of milliseconds while a tone is pres^ 
diode function generators (VDFGs), which produce an ented corresponding to each time the edge of the pat- 
output voltage which is a function of the horizontal po- 30 tern is encountered. 

sition of the scanning dot of oscilloscope 10. An in- Referring to FIG. 4, an alternate embodiment of the 
verter 32 is connected in front of the input to function invention is shown. This embodiment is similar to that 
generator 30 so that the input thereto is the inverse of of FIG. 1 and like elements have been given the same 
that of function generator 28. The output voltages of numbers with primes attached. In this embodiment, the 
VDFGs 28 and 30 are shown in FIG. 2, with curve 28A 35 image pattern or picture is scanned by the raster pat- 
corresponding to the output of generator 28 and curve tern generated by a modified slow scan TV camera 50 
30A corresponding to output of generator 30. These such as a Robot Research Model 80. The camera 50 is 
voltages are preferably roughly exponential functions . modified to produce 0 to 4-10 volt D.C. signals repre- 
of the horizontal position of the scanning dot because sentative of the vertical and horizontal position of the 
the ear responds approximately logarithmically to the 40 scan. In an exemplary embodiment, a scan lasting 8 
sound intensity. seconds and containing 30 horizontal lines was used. A 

Vertical sawtooth generator 14 also performs a sec- further D.C. voltage is produced which is proportional 
ond function in addition to controlling the vertical posi- to the light intensity at each point in the scan and hence 
tion of the scanning dot, the output of vertical sawtooth generally corresponds to the output of the photomulti- 
generator 14 being used to control the frequency of a plier of FIG. 1. 

voltage controlled oscillator ( VCO) 36. More specifi- To eliminate extra blanking pulses generated by cam- 
cally, the output of sawtooth generator 14 is connected era 50, a position sensor 52 is connected to the hori- 
to a further function generator 34 which can also be a zontal output and is used to control switching of a fur- 
VDFG and which, similarly to VDFGs 28 and 30, pro- ther electronic switch 54. The input of switch 54 is con- 
duces an output voltage which is an exponential nected to the intensity output of camera 50 so that 
function of the vertical position of the scanning dot. transmission of this output is controlled in accordance 
This voltage controls the frequency output of VCO 36, with the horizontal position of the scan, 
an exponential voltage being used because, to a first ap. The output of switch 54 is differentiated by differen 
proximation, the ear responds logarithmically to fre- tiator 56 the output of which is, in turn, processed by 
quency. By varying the frequency of VCO 36 in accor- an absolute value amplifier 58 and a pulse stretcher 60 
dance with the vertical position of the scan, the resul- before application to electronic switch 26'. The re- 
tant sound heard by a subject is made to seem to come mainder.of the system is similar to that of FIG. 1. Con- 
front a corresponding position in the scanned pattern. . sidering the operation of the systems of the invention, 
Thus, high frequency tones seem to originate from high 6Q pictures or image patterns are presented to subjects are 
positions in the scan and lower frequency tones seem scanned with a raster scan using the oscilloscope 10 of 
to originate from lower positions. See Roffler et al., FIG. 1 or the television camera 50 of FIG. 4. It will be 
Factors That Influence the Localization of Sound in the appreciated that raster scanning refers to* scanning 
Vertical Plane. JASA 43, 1255-1259, 1968 and Roffler from top to bottom and left to right much as one reads 
et al. t Localization of Tonal Stimuli in the Vertical Plane, 6$ print on a page. In general terms, when the scan crosses 
JASA 43, 1260-1 266, 1968. a part of the object to be presented, switch 26. or 26' 

Switch 26 controls the connection of the output. of is actuated and a sound is heard by the subject. The 
VCO 36 to first and second analog multipliers 38 and pitch of the sound is high when the scan is near the top 
40. With switch 26 turned on, the outputs of VDFGs 28 of the picture and as the scan approaches the bottom, 
J™? 30 are m ultipli ed by the audio output of VC O 36 the pitch decreases exponentially. If the part of the pat' 
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tern is on the left, the volume is louder in the left ear- 
phone of headphones 44 and hence the sound source 
appears to be on the left. Similarly, if the part of the 
pattern is on the right the volume will be louder for the 
right earphone. Although it may be difficult to under- 
stand what the sounds are like without hearing them, a 
description of the sounds associated with a particular 
pattern might be helpful. Thus, the sounds representing 
the outline of a triangle as scanned by the system in 
FIG. 1 would begin with a single high frequency tone 
burst of equal volume in both ears (assuming the trian- 
gle is centered) corresponding to the scan hitting the 
apex of the triangle. As the scan progresses downward, 
the frequency of the tones decrease and almost imme- 
diately two distinct tone bursts are presented, one 
toward the left and one toward the right. The tones 
seem to grow farther apart with each new horizontal 
scan line until the bottom side of the triangle is 
reached. At this time, the tone stays on continuously as 
the scan goes from the far left to the far right. Further, 
the volume, gradually decreases in the left earphone 
and gradually increases in the right earphone during 
this scan. 

It should be noted that signals other than variable 
amplitude signals can be used to indicate the horizontal 
position of the scan and that, for example, signals de- 
layed with respect to one another or differing in phase 
can be used to produce the same effect. Further, an 
array of solid state light detectors such as photodiodes 
might be substituted for the television and oscilloscope 
scanning arrangements described hereinabove. 

Experimental testing of the invention has proved very 
satisfactory, the "coding" of the figures or patterns 
presented being natural and easy to understand. In fact, 
nine and ten year old blind children have learned to 
identify patterns as complex as nine and eleven pointed 
starts with only 4 hours of training. The invention, par- 
ticularly where a portable television camera is used 
corresponding to that discussed above in connection 
with FIG. 2, can also be used as a mobility aid to pres- 
ent an auditory "picture** of a room to a blind person 
so that he can "see" the positions of the furniture in the 
room. Because an auditory rather than tactile display 
is provided the system can also be used to represent the 
spatial pattern of Braille dots to blind people with in- 
jured hands, the aged blind and the diabetic blind. 
These last two groups may include many people who 
have their sense of touch so impaired by age or disease 
that they cannot read ordinary Braille. These uses are, 
of course, exemplary and are not intended to be ex- 
haustive. 

Although the invention has been described relative to 
exemplary embodiments thereof, it will be understood 
that variations and modifications can be effected in 
these embodiments without departing from the scope 
and spirit of the invention. 

I claim: 

1. A system for producing a. two-dimensional audi- 
tory representation of an image pattern, said system 
comprising 

scanning means for scanning the image pattern; 

means for generating a variable frequency signal the 
frequency of which varies in accordance with the 
vertical position of the scan; 

means for generating separate electrical signals for 
each ear of a user which vary in amplitude in accor- 
dance with the horizontal position of said scan; 
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means for combining said, variable frequency signal 
and said separate electrical signals into corre- 
sponding auditory signals for each ear of a user rep- 
resentative of the vertical and horizontal position 
5 . of the scan; 

means for generating a control signal related to the 
image pattern scanned by said scanning means; and 

means for controlling the auditory signals heard by 
10 the user in accordance with, said control signal. 

2. A system as claimed in claim 1 further comprising 
headphones connected to the output of said combining 
means and including left and right earphones, said 
means for generating separate electrical signals includ- 

15 ing means for producing a first variable amplitude sig- 
nal and means for generating a second variable ampli- 
tude signal, said first variable signal being connected to 
said left earphone and said second variable amplitude 
signal being connected to said right earphone through 

20 said combining means. 

3. A system as claimed in claim 2 wherein said com- 
bining means comprises analog multiplier means. 

4. A system as claimed in claim 2 wherein said first 
and second variable amplitude signal generating means 

25 respectively comprise first and second function genera- 
tor means for generating a signal whose amplitude var- 
ies approximately exponentially with the horizontal po- 
sition of the scan, one of said first and second variable 
amplitude signals being the inverse of the other. 

30 5. A system as claimed in claim 4 wherein said func- 
tion generator means each comprise a variable diode 
function generator. 

6. A system as claimed in claim 4 wherein said vari- 
able frequency signal generating means includes a volt- 

35 age controlled oscillator. 

7. A system as claimed in claim 6 wherein said vari- 
able frequency signal generating means further com- 
prises further function generator means for generating 
an output signal the amplitude of which varies approxi- 

40 mately exponentially with the vertical position of the 
scan and which is connected to the input of said voltage 
controlled oscillator. 

8. A system as claimed in claim 7 wherein said first 
and second function generator means and said further 
function generator means each comprise variable diode 
function generators. 

9. A system as claimed in claim 7 wherein said scan- 
. ning means comprises a television camera including 

50 means for generating a first D.C. voltage whose ampli- 
tude varies in accordance with the vertical position of 
the scan, means for generating a second D.C, voltage 
whose amplitude varies in accordance with the hori- 
zontal position of the scan and a third D.C. voltage 

55 whose amplitude varies in accordance with the inten- 
sity of the scan, said system further comprising means 
for processing said third D.C. voltage to produce said 
control signal. 

10. A system as claimed in claim 9 wherein said sig- 
nal processing means comprises differentiator means, 
said means for controlling the auditory signals heard by 
the user comprising an electronic switch controlled by 
said control signal and connected to the output of said 
variable frequency oscillator. 

65 1 1 . A system as claimed in claim 7 wherein said scan- 
ning means comprises an oscilloscope, a vertical saw- 
tooth generator for generating a vertical control volt- 
age for controlling the vertical scan position of said os- 
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cilloscope, a horizontal sawtooth generator for generat- 
ing a horizontal control voltage for controlling the hori- 
zontal scan position of said oscilloscope, the output of 
said horizontal sawtooth generator being connected to 
the inputs of said first and second function generator 5 
means and the output of said vertical sawtooth genera- 
tor being connected to the input of said further func- 
tion generator; photomultiplier tube means for receiv- 
ing light from said oscilloscope as modulated by the 
image pattern; and level sensor means connected to the 10 
output of said photomultiplier tube means for produc- 
ing said control signal. 

12. A system as claimed in claim 11 wherein said 
means for controlling the auditory signals heard by the 
user comprises electronic switch connected to the out- 15 
put of said level sensor meand and to the output of said 
voltage controlled oscillator. 

13. A system.as claimed in claim 1 1 further compris- 
ing means for connecting the output of said level sensor 
means back to the inputs of said vertical and horizontal 20 
sawtooth generators for controlling the scanning rate of 
said oscilloscope in accordance with the output of the 
level sensor means. 

14. A system as claimed in claim 1 wherein said com- 
bining means comprises first and second analog multi- 25 
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plier means, said means for controlling the auditory sig- 
nals heard by the user comprising electronic switch 
means responsive to said control signal for controlling 
connection of said variable frequency signal to the : 
input of said analog multiplier means. 

15. A system for producing a two-dimensional audi- 
tory representation of an image pattern, said system 
comprising 

scanning means for scanning the image pattern; 

means for generating a variable frequency signal the 
frequency of which varies in accordance with the 
vertical position of the scan; 

means for generating separate electrical signals for 
each ear of a user which vary in phase in accor- 
dance with the horizontal position of said scan; 

means for combining said variable frequency signal 
and said separate electrical signals into corre- 
sponding auditory signals for each ear of a user rep- 
resentative of the vertical and horizontal position 
of the scan; 

means for generating a control signal related to the 
image pattern scanned by said scanning means; and 

means for controlling the auditory signals heard by 
the user in accordance with said control signal, 
.* * * ' . * * • 
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